In recent years, a digital camera has become very popular as a camera. In the digital camera, an image is taken using an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) and the image is displayed on a monitor such as a LCD (Liquid Crystal Display) disposed on the digital camera such that a user can view the image (hereinafter, such an image taken by the imaging device will be referred to simply as a captured image). An image signal of the captured image is converted into digital form and transferred to an image processing apparatus such as a personal computer (PC) via a storage medium such as a flash memory or via wired or wireless communication using a cable or an infrared ray. On receiving the image data captured by the digital camera, the personal computer displays the image on a monitor such as a CRT (Cathode Ray Tube) or a LCD (Liquid Crystal Display) so that a user can view the image and edit it.
When an image of a subject that is not well lighted is taken by a digital camera in a darkish environment such as a shaded place or a poorly lighted room, the shutter speed should be set to be low enough to obtain sufficient exposure (that is, the exposure time should be long enough).
When the exposure time is long, if the digital camera is firmly held, for example, by placing it on a tripod such that the digital camera does not shake, a good image can be obtained. However, in a case in which the digital camera is held in hands, the digital camera can shake. If the camera shakes when the shutter is in an open state (during a shutter time), a resultant captured image of a subject is blurred. The blurred image is called a “hand shake blurred” image or a “camera shake blurred” image.
It is known to obtain equivalently sufficient exposure by simply cumulatively superimposing two or more captured images, instead of decreasing the shutter speed (increasing the shutter time) (an example of such a technique may be found, for example, in Japanese Unexamined Patent Application Publication No. 05-236422).
However, in the technique disclosed in Japanese Unexamined Patent Application Publication No. 05-236422, because two or more captured images are simply superimposed, an image is blurred if the digital camera shakes, as in the case in which an image is captured at a low shutter speed.
In some digital cameras available from Canon Kabushiki Kaisha, a technique called an image stabilizer (IS) is used to prevent an image from being blurred when a digital camera shakes.
In the technique using the image stabilizer, a shake or vibration of a digital camera is detected by a pre-sensor disposed in an optical lens system, and a part (correction optical system) of the lens system is moved in a direction perpendicular to the optical axis in accordance with a digital signal indicating the detected shape or vibration of the digital camera so that refraction of light is changed to prevent the image from being blurred.
The image stabilizer makes it possible to cancel out blurring of an image due to shaking of a camera or a stage on which the camera is placed caused by shaking of a hand or a wind, which is significant in particular when an image is taken at a low shutter speed by a digital camera held in hands, thereby allowing a user to obtain a sharp image.
However, the image stabilizer needs a dedicated sensor responsible for sensing shaking of the camera and also needs a mechanism for quickly moving a part (correction optical system) of the lens system, and thus the image stabilizer has a complicated structure, which results in an increase in production cost.
Another known technique to prevent an image from being blurred when the image is captured by a shaking camera is to successively capture a plurality of images at a high shutter speed, detect the displacement of second and following captured images relative to a first captured image, compensate for the displacement of the second and following captured images, and superimposing the compensated images on the first captured image (examples of this technique may be found, for example, in Japanese Unexamined Patent Application Publications Nos. 2000-217032, 2000-224460, 2000-244803, 2000-244797, 2000-069352, 10-341367, and 09-261526). Basically, in this conventional technique, images similar to a first captured image are produced from second and following captured images by means of correction or interpolation, and the resultant images are simply added or linearly added on a pixel-by-pixel basis to the first captured image.
In this technique, each of images successively captured at a high shutter speed has practically no blur although images are dark. An image finally obtained by superimposing of second and following captured images on a first image can have high brightness similar to that obtained when the exposure time is set properly.
In this technique, the correction of the detected displacement of the second and following captured images relative to the first captured image is made by interpolating color signals (data) including a R (Red) signal (red data), a G (Green) signal (green data), and a B (Blue) signal (blue data) for each pixel, by using an interpolation function based on linear interpolation, bicubit interpolation, or the like.
However, in the case of an imaging device using a single-plane sensor as an imaging device, each pixel outputs only one of color signals, that is, an R signal, a G signal, or a B signal. This makes it impossible to correct displacements of a plurality of images by means of interpolation described above, and any interpolation method applicable to the single-plane sensor is known.